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DESCRIPTION 
INFORMATION PROCESSING APPARATUS 
Technical Field 

The present invention relates to an 
information processing apparatus, in particular, to an 
information processing apparatus that can execute a 
plurality of signal processes. 
Background Art 

A signal processing device disposed in a 
signal 1 processing apparatus is normally structured so 
that one function is accomplished. 

In other words, to accomplish a plurality of 
different signal processes, it is necessary to provide 
devices corresponding to the number of required 
processes. Thus, for example, it was difficult to 
compactly structure and reduce the cost of the 
apparatus . 

Disclosure of the Invention 

The present invention was made from the 
above-point of view. According to the present 
invention, corresponding to a predetermined function 
execution request, the internal structure and process 
contents of the same hardware are changed so as to 
execute a plurality of processes. 

An information processing apparatus of the 
present invention, comprising a plurality of 
classifying adaptive processing circuits for performing 



a classifying adaptive process for input information 
signals, and a switching circuit for switching a 
connection relation among the plurality of classifying 
adaptive processing circuits. 
5 According to the information processing 

apparatus of the present invention, a plurality of 
classifying adaptive processing circuits perform a 
classifying adaptive process for a desired function 
corresponding to an input information signal. The 
10 relation of the connections of the plurality of 

« classifying adaptive processing circuits is switched. 

v5 At least one of the classifying adaptive 

Q processing circuits is configured for switching the 

w corresponding classifying adaptive process for the 

715 corresponding information signal as the connection 

L relation of the switching circuit is switched. 

J=X At least one of the plurality of classifying 

^; adaptive processing circuits is configured for 

switching the structure of the corresponding 
20 classifying adaptive process as the connection relation 

of the switching circuit is switched. 

The structure represents a structure of class 
taps or a structure of predictive taps. 

At least one of the plurality of classifying 
25 adaptive processing circuits is configured for 

switching a coefficient of the corresponding 
classifying adaptive process so as to switch the 
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process for the corresponding information signal as the 
connection relation is switched by the switching 
circuit . 

The input information signals are output 
through the plurality of classifying adaptive 
processing circuits. 

A pre-processing circuit is configured for 
switching a predetermined process as the connection 
relation is switched- The pre-processing circuit is 
configured for performing a predetermined process for 
the corresponding input information signal and 
inputting the processed result to the classifying 
adaptive processing circuit. 

A post -processing circuit switches a process 
as the relation of the connections is switched. The 
post -processing circuit performs a predetermined 
process for an output of a classifying adaptive 
processing circuit. 

Examples of processes that the classifying 
adaptive processing circuits accomplish are as follows. 

The information signals are picture data 
composed of pixel information. One of the plurality of 
classifying adaptive processing circuits is configured 
for performing the classifying adaptive process 
corresponding to the pixel information of the 
corresponding input information signal and predicting 
pixel information that has to be present between the 



pixel information of the input information signal and 
pixel information adjacent thereto so as to improve the 
resolution of the picture data. 

The information signals are picture data 
5 composed of pixel information. One of the plurality of 

classifying adaptive process circuits is configured for 
performing the classifying adaptive process for the 
corresponding input information signal using a prepared 
left eye coefficient and predicting pixel information 
10 of left-eye picture data and for performing the 

1 classifying adaptive process for the corresponding 

input information signal using a prepared right -eye 
i coefficient and predicting pixel information of right - 

j eye picture data so as to generate stereo picture data 

"15 with the left-eye picture data and the right-eye 

I picture data* 

z z The information signals are picture data 

} composed of pixel information. One of the plurality of 

classifying adaptive processing circuits is configured 

20 for performing the classifying adaptive process for the 

corresponding input information signal using a prepared 
luminance signal coefficient and predicting a luminance 
signal component of the picture data and another one of 
the plurality of classifying adaptive processing 

25 circuits is configured for performing the classifying 

adaptive process using prepared color difference signal 
coefficients and predicting color difference components 
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of the picture data so as to separate the picture data 
into the luminance component and the color difference 
components . 

The information signals are picture data 
5 composed of pixel information. At least two of the 

plurality of classifying adaptive processing circuits 
are configured for performing the classifying adaptive 
process for the pixel information having different 
phases and changing the number of pixel information 

10 that composes the picture data. 

The information signals are picture data 
composed of pixel information. At least two of the 
plurality of classifying adaptive processing circuits 
are configured for performing the classifying adaptive 

15 process and obtaining a plurality of picture data 

having different resolutions corresponding to the 
classifying adaptive process performed by the plurality 

\ °f classifying adaptive processing circuits. 

One of the plurality of classifying adaptive 

20 processing circuits is configured for performing the 

classifying adaptive process for the corresponding 
input information signal and obtaining picture data 
having a first resolution and another one of the 
plurality of classifying adaptive processing circuits 

25 is configured for performing the classifying adaptive 

process for picture data having the first resolution 
and obtaining picture data having a second resolution. 
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The information signals are picture data 
composed of pixel information and structured in the 
unit of a frame. One of the plurality of classifying 
adaptive processing circuits is configured for 
5 performing the classifying adaptive process for the 

corresponding information signal that is input in the 
unit of a frame and generating picture data of frames 
chronologically preceded and followed by a frame of the 
input information signal- ~ 
10 Brief Description of Drawings 

y Fig. 1 is a block diagram showing an example 

of a basic structure of a data processing apparatus 
O according to the present invention; Fig. 2 is a block 

hi diagram showing an example of the structure of a pre- 

s 15 processing portion of the data processing apparatus; 

u Figs . 3A and 3B are block diagrams showing an example 

of the structure of a data processing portion of the 
— data processing apparatus; Fig. 4 is a block diagram 

showing an example of the structure of a post- 
20 processing portion of the data processing apparatus; 

Fig. 5 is a block diagram showing an example of the 
structure of a data processing apparatus 1 that 
executes a process for increasing the resolution in 
process mode (1); Fig. 6 is a schematic diagram showing 
25 pixel arrangements of an SD picture and an HD picture; 

Figs . 7A and 7B are block diagrams showing an example 
of the connections of the data processing portion in 
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the process mode (1); Fig. 8 is a block diagram showing 
an example of the structure of a classifying adaptive 
processing portion that performs a prediction process; 
Fig. 9 is a schematic diagram showing an example of 
class tap extraction information in the process mode 
(1); Fig. 10 is a schematic diagram showing other pixel 
arrangements of an SD picture and an HD picture; Fig. 
11 is a block diagram showing an example of the 
structure of a learning device _that performs a learning 
process for calculating predictive coefficients that 
are pre- stored in a coefficient memory of a classifying 
adaptive processing portion; Fig. 12 is a block diagram 
showing an example of the structure of a data 
processing apparatus that performs another processing 
method for the process mode (1); Figs. 13A and 13B are 
block diagrams showing an example of the connections of 
the data processing portion that performs the other 
processing method for the process mode (1); Fig. 14 is 
a schematic diagram showing an example of class tap 
extraction information in the other processing method 
for the process mode (1); Fig. 15 is a schematic 
diagram showing a pixel arrangement in the other 
processing method for the process mode (1); Fig. 16 is 
a block diagram showing an example of the structure of 
the data processing apparatus that executes a process 
for generating a picture dedicated for a left eye and a 
picture dedicated for a right eye in process mode (2); 



Figs* 17A and 17B are block diagrams snowing an example 
of the connections of the data processing portion in 
the process mode (2); Fig. 18 is a schematic diagram 
showing an example of class tap extraction information 
in the process mode (2); Fig. 19 is a schematic diagram 
showing in more reality predictive taps in the process 
mode (2) ; Fig. 20 is a schematic diagram for explaining 
a picture that is learnt; Fig. 21 is a block diagram 
showing an example of "the structure of the classifying 
adaptive processing portion that executes a learning 
process in the process mode (2); Fig. 22 is a block 
diagram showing an example of the structure of the data 
processing apparatus that executes a process for 
generating a luminance signal and color difference 
signals in process mode (3); Figs. 23A and 23B are 
block diagrams showing an example of the connections of 
the data processing portion in the process mode (3); 
Fig. 24 is a block diagram showing an example of the 
structure of the data processing apparatus that 
accomplishes a Y/C separating device that separates a 
composite signal NTSC into a component signal ( Y, U, V) 
corresponding to the classifying adaptive process in 
the process mode (3); Fig. 25 is a schematic diagram 
showing an example of class tap extraction information 
in the process mode (3); Fig. 26 is a schematic diagram 
showing an example of the predictive tap extraction 
information in the process mode (3); Fig. 27 is a block 
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diagram showing an example of the structure of the 
classifying adaptive processing portion that executes a 
learning process in the process (3) is executed; Fig. 
28 is a block diagram showing an example of the 
5 structure of the data processing apparatus that 

converts the aspect ratio in process mode (4); Figs. 
29A and 29B are block diagrams showing an example of 
the connections of the data processing portion in the 
process mode (4); Fig. 30 is a schematic diagram for 
10 explaining the classifying adaptive processing portion 

3 of the input data processing portion that performs a 

predictive process in the process mode (4); Fig. 31 is 
^ a block diagram showing an example of the structure of 

^ the data processing apparatus that generates a picture 

J 15 having a different resolution in process mode (5); Figs. 

32A and 32B are block diagrams showing an example of 
il the connections of the data processing portion 12 in 

j the process mode (5); Fig. 33 is a block diagram 

showing an example of the structure of the data 
20 processing apparatus that converts a frame rate in 

process mode (6); Figs. 34A and 34B are block diagrams 
showing an example of the connections of the data 
processing portion in the process mode (6); Fig. 35 is 
a schematic diagram for explaining frame structures of 
25 an input picture and an output picture; and Fig. 36 is 

a schematic diagram showing an example of class tap 
extraction information in the process mode (6). 
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Best Modes for Carrying out the Invention 

Next, with reference to the accompanying 
drawings, an embodiment of the present invention will 
be described- Fig. 1 shows an outline of an example of 
5 the structure of a data processing apparatus 1 

according to the present invention. As shown in Fig. 1, 
a picture data as input data is supplied from a picture 
processing device such as a television receiver, a 
video camera, a VTR, a scanner, or a digital camera to 
10 the data processing apparatus 1 according to the 

present invention. 
□ The data processing apparatus 1 according to 

;3 the embodiment can execute process modes that 

,U accomplish for example the following contents 

15 corresponding to requests. Picture data that is 

=i generated as the processed result of each process mode 

■% is output as output data to an external device (for 

f example, a displaying device or a recording and 

reproducing device ) . 
20 (1) Increase of resolution (referred to as 

process mode (1) ) , 

(2) Generation of a picture dedicated for 
right eye and a picture dedicated for left eye 
(referred to as process mode (2)), 
25 (3) Generation of a luminance signal and 

color difference signals (referred to as process mode 
(3)), 
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(4) Change of aspect ratio (referred to as 
process mode ( 4 ) ) , 

(5) Generation of a picture having a 
different resolution (referred to as process mode (5)), 
and 

(6) Conversion of frame rate (referred to as 
process mode (6) ) . 

The details of the processes of those process 
modes performed by the data processing apparatus 1 will 
be described later. 

The data processing apparatus 1 has a 
function controlling portion 11, a data processing 
portion 12, a pre-processing portion 13, and a post- 
processing portion 14. A command that causes a desired 
process to be executed is input to the function 
controlling portion 11. The function controlling 
portion 11 supplies a control signal corresponding to 
the input command to the data processing portion 12 so 
as to initialize the data processing portion 12 and 
executes a process corresponding to the command. In 
addition, the control signal that is output from the 
function controlling portion 11 is supplied to the pre- 
processing portion 13 and the post -processing portion 
14. The control signal causes the pre-processing 
portion 13 and the post-processing portion 14 to 
execute predetermined processes. 

The pre-processing portion 13 performs a 



predetermined pre-process for picture data as input 
data corresponding to the control signal supplied from 
the function controlling portion 11 and supplies the 
processed data to the data processing portion 12. In 
addition, the control signal is supplied from the 
function controlling portion 11 to the data processing 
portion 12. The data processing portion 12 is 
initialized with the control signal. The data 
processing portion 12 performs a classifying adaptive 
process corresponding to a process designated with the 
control signal for the supplied input data and outputs 
data generated as the processed result. An output of 
the data processing portion 12 is supplied to the post- 
processing portion 14. The post -processing portion 14 
performs a predetermined post-process for the output of 
the data processing portion 12 corresponding to the 
control . signal supplied from the function controlling 
portion 11 and outputs the result as output data to an 
external device . 

Fig. 2 shows an example of the structure of 
the pre-processing portion 13. The pre-processing 
portion 13 has selectors 130 arid 132 and a plurality of 
delaying circuits 131A, 132B, and so forth. A control 
signal that is supplied from the function controlling 
portion 11 is input to the selectors 130 and 132 so as 
to control the selections of the signal paths of the 
selectors 130 and 132. The delaying circuits 131A, 



131B, and so forth have different delay times. For 
example , the delaying circuit 131A delays an input 
signal in the unit of a pixel. The delaying circuit 
131B delays an input signal in the unit of a line. The 
delaying circuit 131C delays an input signal in the 
unit of a frame. Signals that are output from the 
delaying circuits 131A, 131B, 131C, and so forth are 
input to the selector 132. The selector 132 selects 
the signals that are output from the delaying circuits 
131A, 131B, 131C r and so forth corresponding to the 
control signal supplied from the function controlling 
portion 11. The signals selected by the selector 132 
are output from the pre-processing portion 13. 

Alternatively, in the pre-processing portion 
13, the signals that are output from the selector 130 
may be directly input to the selector 132 not through 
the delaying circuits 131A, 13 IB, 13 1C, and so forth. 
Further alternatively, the selector 130 may output the 
same signal in parallel as a plurality of outputs. 
Further alternatively, the pre-processing portion 13 
may have a processing circuit other than the delaying 
circuits 131A, 131B, and so forth. 

Fig. 3 shows an example of the structure of 
the data processing portion 12. The data processing 
portion 12 is composed of an input data processing 
portion 21, a selector 22, an intermediate data 
processing portion 23, an output data processing 
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portion 24, and a selector 25. 

The input data processing portion 21 has a 
plurality of classifying adaptive processing portions 
(in this example, four classifying adaptive processing 
portions 31-1 to 31-4 (hereinafter simply referred to 
as classifying adaptive processing portion 31 when it 
is not necessary to distinguish them) (such notation 
applies to other structural portions)* The classifying 
adaptive processing portion 31 executes a classifying 
adaptive process for supplied data (namely, input data 
that is input to the data processing apparatus 1 or 
data that is input to the data processing apparatus 1 
and pre-processed by the pre-processing portion 13) and 
supplies the generated data as the processed result to 
the selector 22. 

The data that is processed (generated) in the 
input data processing portion 21 and the intermediate 
data processing portion 23 is supplied to the selector 
22. The select portion 22 selects a destination for 
the supplied data (the destination is for example a 
memory 41 of the intermediate data processing portion 
23 or a classifying adaptive processing portion 51 of 
the output data processing portion 24) and supplies the 
data to the selected destination. 

The intermediate data processing portion 23 
has a plurality of memories (in this example, four 
memories 41-1 to 41-4). The intermediate data 
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processing portion 23 performs a write control and a 
read control of the memory 41 for data supplied from 
the selector 22. For example, the intermediate data 
processing portion 23 executes a process for changing a 
horizontal scanning sequence to a vertical scanning 
sequence. Data that is read from the memory 41 of the 
intermediate data processing portion 23 is returned to 
the selector 22. 

The output data processing portion 24 has a 
plurality of classifying adaptive processing portions 
(in this example, eight classifying adaptive processing 
portions 51-1 to 51-8). The classifying adaptive 
processing portion 51 executes a classifying adaptive 
process for the data supplied from the selector 22 
corresponding to the control signal received from the 
function controlling portion 11 and outputs the 
generated data as the processed result. Data that is 
output from the classifying adaptive processing 
portions 51-1 to 51-8 is input to the selector 25. The 
selector 25 selects data corresponding to the control 
signal that is supplied from the function controlling 
portion 11 and outputs the selected data. 

Fig. 4 shows an example of the structure of 
the post -processing portion 14. The post -processing 
portion 14 has a selector 140, a plurality of 
processing portions 141A, 141B, and so forth, and a 
selector 142. Data that is output from the data 



processing portion 12 is input to the selector 140. 
The selector 140 switches paths of the input data 
corresponding to a control signal that is supplied from 
the function controlling portion 11 and inputs the 
5 switched data to the plurality of the processing 

portions 141A, 141B, and so forth. 

According to the embodiment, the processing 
portion 14 1A is a line sequence converting portion that 
performs a conversion in the unit of a line. The 
10 processing portion 141B is a scanning direction 

^ converting portion that performs a conversion in the 

3 unit of a pixel. The processing portion 141C is a 

j multiplexing portion that multiplexes data. The 

J processing portions 141A, 141B, and so forth process 

15 input data corresponding to a control signal supplied 

h from the function controlling portion 11. 

Z Next, the structure and operation of the data 

processing apparatus 1 that executes the processes for 
accomplishing the above- described process modes (1) to 
20 (6) will be described. 

Fig. 5 shows an example of the structure of 
the data processing apparatus 1 that executes the 
process for increasing the resolution in the process 
mode (1). In this example, an SD (Standard Density) 
25 picture (for example, picture data in an interlace 

format of which the number of lines is 525) 
(hereinafter, sometimes referred to as SD picture data) 
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is input as input data from an external device to the 
data processing apparatus 1. It is assumed that the 
data processing apparatus 1 generates an HD (High 
Density) picture (for example, picture data in a 
5 progressive format of which the number of lines is 525 

and of which the number of pixels in the horizontal 
direction is twice that of the SD picture) (hereinafter, 
sometimes, the HD picture data is referred to as HD 
picture data ) ♦ 

10 In this example, a command that causes the 

q process for converting an SD picture into an HD picture 

^5 to toe executed is additionally input to the function 

g controlling portion 11. The function controlling 

:]] portion 11 outputs a control signal corresponding to 

-15 the input command to the data processing portion 12. 

The data processing portion 12 generates data 
ru of each line of the HD picture with the SD picture that 

O is input as the input data to the data processing 

apparatus 1 corresponding to the control signal 
20 received from the function controlling portion 11. 

Accurately speaking, the data processing portion 12 
predicts pixels arranged on each line. 

In reality, as shown in Fig. 6, when a 
predetermined field of an SD picture is composed of 
25 lines Lsp-x, L S d-2» Lsd-3, and so forth on which pixels 

denoted by large circles O (hereinafter referred to as 
SD pixels) are arranged, the data processing portion 12 
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generates lines L H d-i» I»hd-2. Lhd-3 on which pixels of an 
HD picture denoted by small circles O are arranged at 
the same positions as the lines L S d-i, L S d-2* I«sd-3* and so 
forth of the SD picture (hereinafter, sometimes, a line 
of an HD picture at the same position as a line of an 
SD picture is referred to as line A) and lines Lhd-2, 
Ls25p-4# and so forth of the HD picture at positions that 
are not same as the lines of the SD picture 
(hereinafter, a line of an HD picture at a position 
that is not same as a line of an SD picture is referred 
to as line B) . 

The data processing portion 12 separately 
generates pixels of line A and pixels of line B. In 
the post-processing portion 14 disposed downstream of 
the data processing portion 12, the selector 140 
selects the line sequence converting portion 141A and 
outputs line A and line B supplied from the data 
processing portion 12 in line sequence. In other words, 
HD picture data is output as output data from the line 
sequence converting portion 14 1A to an external device 
through the selector 142. 

In other words, in this example, the data 
processing apparatus 1 non-interlaces an SD picture as 
an interlace format picture, doubles the number of 
pixels in the horizontal direction, and thereby 
generates an HD picture having pixels four times larger 
than an SD pixel. 
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Fig. 7 shows an example of the connections of 
the data processing portion 12 that executes the 
process for increasing the resolution in the process 
mode (1). In this example, in the input data 
processing portion 21, the two classifying adaptive 
processing portions 31-1 and 31-2 are used. SD picture 
data that is input to the data processing apparatus 1 
is supplied as input data to the classifying adaptive 
processing portions 31-1 and 31-2. 

The classifying adaptive processing portion 
31-1 executes a classifying adaptive process for 
generating line A (in the example shown in Fig. 6, L H d-i 

lnu>-5r and so forth) that compose an HD picture 
with supplied SD picture data. In other words, in the 
classifying adaptive process, a process for predicting 
HD pixels arranged on line A is performed. The 
classifying adaptive processing portion 31-2 executes a 
classifying adaptive process for generating line B (in 
the example shown in Fig. 6, Lhd-2, Lhd-4, and so forth) 
that compose an HD picture with an SD picture. In 
other words, in the classifying adaptive process, a 
process for predicting HD pixels arranged on line B is 
performed. 

Pixel data of line A and line B generated by 
the classifying adaptive processing portions 31-1 and 
31-2 is supplied to the select portion 22. 

The select portion 22 supplies pixel data of 
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line A and line B simultaneously supplied from the 
classifying adaptive processing portions 31-1 and 31-2 
of the input data processing portion 21 to the post- 
processing portion 14. In the post-processing portion 
5 14, the selector 140 selects the line sequence 

converting portion 141A corresponding to a control 
signal supplied from the function controlling portion 
11. Thus, line A and line B are supplied to the line 
sequence portion 141A. Line A and line B that compose 
10 the HD picture that are simultaneously supplied to the 

O line sequence converting portion 13 are converted into 

Ku a progressively scanned picture in the line sequence 

O and output to an external device. 

hi The intermediate data processing portion 23 

~15 and the output data processing portion 24 are not used 

L in the process mode. 

Fig. 8 shows an example of the structure of 
**f the classifying adaptive processing portion 31-1 of the 

input data processing portion 21. As was described 
20 above, the classifying adaptive processing portion 31-1 

performs the process for line A. The classifying 
adaptive processing portion 31-1 comprises a 
classifying portion 61, an adaptive processing portion 
62, and a register group 63. The classifying portion 
25 61, the adaptive processing portion 62, and the 

register group 63 select a bank for the process mode 
corresponding to a control signal that is output from 
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the function controlling portion 11 to the data 
processing portion 12 and execute various processes 
corresponding to a coefficient set of the selected bank. 

The classifying portion 61 is composed of a 
5 class tap extracting circuit 71 and a classifying 

circuit 72. Successively designating HD pixels on line 
A of an HD picture as considered pixels, the 
classifying portion 61 classifies the considered pixels 
as predetermined classes . 
10 In other words, the class tap extracting 

j circuit 71 selects SD pixels (hereinafter, sometimes 

Pj referred to as class taps) for each considered pixel 

^ from an SD picture, extracts the class taps, and 

" supplies them to the classifying circuit 72. 

-15 A register 63A of the register group 63 

- registers information (hereinafter referred to as class 

U tap extraction information) of a selected pattern of SD 

j pixels as class taps for each classifying adaptive 

process executed by the classifying adaptive processing 
20 portion 31-1. In other words, before extracting class 

taps, the class tap extracting circuit 71 reads class 
tap extraction information necessary in the example 
(the class tap extraction information corresponds to a 
control signal supplied from the function controlling 
25 portion 11 to the data processing portion 12) and 

extracts class taps corresponding to the class tap 
extraction information. 
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Fig. 9 shows an example of class tap 
extraction information in the process mode (1). In Fig* 
9, • (black circle) and O (white circle) represent SD 
pixels. Assuming that a considered pixel is a black 
5 circle, class taps are composed of 3 pixels x 3 pixels 

that are arranged around the considered pixel and that 
include the considered pixel. Predictive taps (that 
will be described later) are composed of 5 pixels x 5 
pixels wider than class taps by one pixel. In Fig- 9, 
10 X marks represent HD pixels that are finally formed 

Q corresponding to a considered pixel. As represented in 

\3 area A shown in Fig. 9, four HD pixels are formed with 

Q one considered pixel. As a result, above -de scribed 

bj line B is formed. 

ris Next, with reference to Fig. 10, the process 

will be described in more detail. Fig. 10 is a 
schematic diagram showing the detail of one sequence 

; J containing a considered pixel in the vertical direction 

in Fig. 9. For example, SD pixels denoted by large 
20 circles are arranged in one sequence containing a 

considered pixel of an SD picture of a predetermined 
field (namely, in the vertical direction). On the 
other hand, it is assumed that HD pixels denoted by 
small circles are arranged in one sequence 
25 corresponding to the above -de scribed considered pixel 

of the SD picture. When HD pixel Y x of the SD picture 
is a considered pixel, corresponding to class tap 
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extraction information that is read from the register 
63A, a total of three SD pixels that are SD pixel X 2 
that is the closest to HD pixel Y x (at the same 
position in the example shown in Fig, 10) and two SD 
5 pixels Xi and X 3 vertically adjacent to SD pixel X 2 are 

selected as class taps of the considered pixel (HD 
pixel) Yi* 

Returning to Fig, 8, the classifying circuit 
72 of the classifying portion 61 detects a feature of 
10 SD pixels that compose class taps supplied from the 

3 class tap extracting circuit 71 (for example, the 

distribution of pixel values) and supplies a class code 
i that has been assigned for each feature to the adaptive 

~ processing portion 62 (coefficient memory 83), The 

--45 class code is supplied as an address to the coefficient 

^ memory 83. As a pixel value, for example a luminance 

U value of a pixel can be used. 

3 Since each pixel is assigned eight ^bit data, 

in the example, assuming that each SD pixel is assigned 

20 eight -bit data, the number of classes becomes huge. 

Thus, the required memory capacity increases. 

Thus, actually, when classifying each 
considered pixel, the classifying circuit 72 performs 
the process for decreasing (compressing) the number of 

25 bits of each pixel that composes class taps. As an 

example of the method for the compressing process for 
the number of bits, ADRC (Adaptive Dynamic Range 

23 



Coding) process is known. 

In the ADRC, the maximum pixel value MAX and 
the minimum pixel value MIN are detected from pixels 
that compose a process block (class taps). In addition, 
5 . the difference DR between the pixel value MAX and the 

pixel value MIN (= pixel value MAX - pixel value MIN) 
is calculated. The obtained DR is the dynamic range DR 
of the process block- The pixel value MIN is 
subtracted from each pixel value that composes the 
10 process block. Each subtracted value is divided by 

O DR/2 K . As a result, each pixel value that composes 

class taps as the process block is re-quantized to K 
p bits that are smaller than the originally assigned bits 

jy (eight bits). For example, when K = 1, in the example 

ji5 shown in Fig. 10, the number of patterns of pixel 

JT values of three SD pixels is (2 1 ) 3 . In comparison with 

5= the case that the ADRC is not performed, the number of 

C3 patterns can be decreased. 

The compressing process of the classifying 
20 circuit 72 is not limited to the ADRC. Instead, 

another compressing process such as vector quantization 
may be used. 

The classifying circuit 72 decides the class 
of the considered pixel corresponding to the K-blt 
25 pixel value of each SD pixel that composes the class 

taps . 

Before classifying the considered pixel, the 
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classifying circuit 72 reads class tap extraction 
information necessary in the process mode from the 
register 63A of the register group 63, extracts the 
class taps corresponding to the class tap extraction 
information, and classifies the considered pixel with 
the extracted class taps. 

The adaptive processing portion 62 is 
composed of a predictive tap extracting circuit 81, a 
predictive calculating circuit 82, and a coefficient 
memory 83, The predictive tap extracting circuit 81 
selects SD pixels of the SD picture supplied to the 
adaptive processing portion 62 as predictive taps and 
supplies the predictive taps to the predictive 
calculating circuit 82. 

A register 63B of the register group 63 
registers information of selected patterns of 
predictive taps of the SD picture (hereinafter, 
sometimes , referred to as predictive tap extraction 
information) for each classifying adaptive process 
executed by the classifying adaptive processing portion 
31-1. In other words, before forming predictive taps, 
the predictive tap extracting circuit 81 reads required 
predictive tap extraction information (predictive tap 
extraction information corresponding to a control 
signal supplied from the function controlling portion 
11 to the data processing portion 12) from the register 
63B and extracts the predictive taps from the input 
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data corresponding to the predictive tap extraction 
information. In the process mode (1), as was described 
with reference to Fig. 9, the predictive taps are 
composed of 5 pixels x 5 pixels arranged around the 
considered pixel. 

The coefficient memory 83 stores predictive 
coefficient sets (that are obtained in the learning 
process that will be described later) for individual 
classes- When a class is supplied from the classifying 
circuit 72 of the classifying portion 61, a predictive 
coefficient set is read from the address corresponding 
to the class and supplied to the predictive calculating 
circuit 82. 

The coefficient memory 83 is composed of a 
plurality of banks. Each bank stores predictive 
coefficient sets corresponding to the individual 
process modes. Each bank stores predictive coefficient 
sets of corresponding types. A register 63C of the 
register group 63 registers bank selection information 
that is information of bank selections corresponding to 
process modes. 

Before reading a predictive coefficient set, 
the coefficient memory 83 reads bank selection 
information (that is coefficient information 
corresponding to a control signal supplied from the 
function controlling portion 11 to the data processing 
portion 12) corresponding to the process mode and sets 
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a bank corresponding to the bank selection information. 
As described above, in the example, since the 
classifying adaptive processing portion 31-1 performs 
the process for line A, the coefficient memory 83 sets 
5 a bank that stores a predictive coefficient set 

corresponding to line A. Predictive coefficients are 
read from an address of the bank corresponding to the 
supplied class and supplied to the predictive 
calculating circuit ,82. 
10 Predictive taps are supplied from the 

3 predictive tap extracting circuit 81 to the predictive 

'g calculating circuit 82. In addition, a predictive 

^ coefficient set is supplied from the coefficient memory 

jl 83 to the predictive calculating circuit 82. 

"15 The predictive calculating circuit 82 

performs a calculation of Expression (1) that is a 
y linear combination model defined by a linear 

3 combination of a predictive coefficient W and a pixel 

value x using a predictive coefficient set (predictive 
20 coefficients w 1# w 2 , and so forth) supplied from the 

coefficient memory 83 and pixel values (pixel values x x , 
x 2 , and so forth) of pixels that compose predictive 
taps supplied from the coefficient memory 83, obtains a 
predictive value E [y] of the considered pixel (HD 
25 pixel) y, and treats the calculated result as the pixel 
value of a HD pixel (HD pixel arranged on line A). 
E [y] = + w 2 x 2 + ... (1) 
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Alternatively, using a non- linear combination 
model, a calculation can be performed. 

As described above, pixel values of 
individual HD pixels arranged on line A of an HD 
5 picture are predicted. Thus, line A is generated. 

The structure of which the classifying 
adaptive processing portion 31-2 performs a predictive 
process as a classifying adaptive process in the 
process mode (1) is basically the same as the structure 
10 of the above-described classifying adaptive processing 

p portion 31-1. In other words, in the classifying 

,g adaptive processing portion 31-2, the class tap 

O extracting circuit 71 and the coefficient memory 83 of 

the classifying portion 61 and the adaptive processing 
~±5 portion 62 extract class taps, predictive taps, and a 

f~ predictive coefficient set corresponding to required 

tap extraction information, predictive tap extraction 
C3 information, and bank selection information. As a 

result, pixel values of HD pixels arranged on line B of 
20 an HD picture (namely, a line at a position that is not 

the same as a line of the SD picture) are predicted. 
For example, the pixel value of HD pixel Y 2 shown in 
Fig. 10 is predicted. As a result, line B is generated. 

Fig. 11 shows an example of the structure of 
25 a learning device that performs a learning process for 

calculating predictive coefficients that are pre-stored 
to the coefficient memory 83 shown in Fig. 8. In the 
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process mode (1), the learning process is performed for 
line A and line B. The learnt results for line A and 
line B are separately output . 

An HD picture as teacher data in the learning 
5 process is supplied to both a thin-out circuit 91 and a 

teacher data extracting circuit 95. The thin-out 
circuit 91 thins out pixels of the HD picture as the 
teacher data so as to generate an SD picture. The 
thin-out circuit 91 supplies the SD picture to a 
10 classifying portion 92 and a predictive tap extracting 

circuit 93* For example, the thin- out circuit 91 
% halves the number of pixels in each of the horizontal 

direction and the vertical direction of the HD picture 
^ so as to form an SD picture, 

3L5 The classifying portion 92 decides the class 

* of the considered pixel in the same process as the 

U classifying portion 61 shown in Fig. 8 and supplies the 

g decided class to address terminals (AD) of a predictive 

tap memory 94 and a teacher data memory 96. The 
20 predictive tap extracting circuit 93 performs the same 

process as the predictive tap extracting circuit 81 
shown in Fig. 8 so as to extract predictive taps. The 
predictive tap extraction circuit 93 supplies the 
extracted predictive taps to the predictive tap memory 
25 94. 

The predictive tap memory 94 stores the 
predictive taps supplied from the predictive tap 
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extracting circuit 93 at an address corresponding to 
the class supplied from the classifying portion 92. 

On the other hand, the teacher data 
extracting circuit 95 extracts an HD pixel as a 
5 considered pixel (designated by the classifying portion 

92 and the predictive tap extracting circuit 93) from 
the supplied HD picture and supplies the HD pixel as 
teacher data to the teacher data memory 96. 

The teacher data memory 96 stores teacher 
10 data supplied from the teacher data extracting circuit 

O 95 at an address corresponding to the class supplied 

i[1 from the classifying portion 92* 

□ The above -de scribed process is performed for 

iuj all pixels as considered pixels that compose an HD 

^L5 picture prepared for the learning process. 

As a result, at the same address of the 
\% teacher data memory 96 or the predictive tap memory 94, 

P an HD pixel of the class corresponding to the address 

as teacher data and SD pixels at the positions of 
20 predictive taps of the HD pixels as learnt data are 

stored. 

In the predictive tap memory 94 and the 
teacher data memory 96, a plurality of pieces of 
information can be stored at the same address* Thus, a 
25 plurality of pieces of learnt data and teacher data 

that are classified as the same class can be stored at 
the same address. 



30 



A calculating circuit 97 reads predictive 
taps as learnt data or HD pixels as teacher data stored 
at the same address of the predictive tap memory 94 or 
the teacher data memory 96, respectively. Using the 
5 predictive taps or HD pixels that are read, by for 

example the method of least squares, a predictive 
coefficient of which the difference between predictive 
value and teacher data becomes minimum is obtained. In 
other words, the calculating circuit 97 solves a normal 
10 equation as Expression (2) for each class so as to 

obtain a predictive coefficient. 

( x " x * ) Wl + ( ^ * ilXi2 ) Wz + + ( 2| x * x ff ) w ' = ( J§ Xi{yi ) 

( % XnXn Y 1 + ( XnXn Y 2 + + ( % XnXij ) Wj " ( 2 * E ) ' * * ( 2 > 

( ^ XijXn ) w * + ( % XijXi2 Y 2 + "* + ( § ^ Y j = ( i x " yi ) 

U The predictive coefficient set that has been 

j obtained in the above- described manner is stored to the 

15 coefficient memory 83 shown in Fig. 8. 

Since the structure of the classifying 
adaptive processing portion 31-2 of the input data 
processing portion 21 that performs the learning 
process is the same as that of the above -described 

20 classifying adaptive processing portion 31-1, the 

description will be omitted. 

In the above -described example, luminance 
values are used as pixel values . When a signal is 
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composed of luminance and color differences, the above- 
described process is performed for each of the 
luminance and color differences so as to generate an HD 
picture. In the case, the classifying adaptive 
5 processing portions 31-3 and 31-4 of the input data 

processing portion 21 may execute a classifying 
adaptive process using color differences and generate 
line A and line B that compose an HD picture. 

The classifying adaptive process for 
10 converting an interlaced picture into a non-interlaced 

^ picture is described in Japanese Patent Application No. 

fl HEI 10-208116 (Japanese Patent Laid-Open Publication No, 

3 2000-41223 laid open on February 8, 2000) and the 

corresponding United State Patent Application (serial 
15 number 09/358272, filed on July 21, 1999) both of which 

=i were filed by the applicant of the present invention. 

JrJ Although the normal equation as Expression 

:=J (2) used in obtaining predictive coefficients has been 

disclosed in prior art (for example, Japanese Patent 
20 Laid-Open Publication No. 2000-41223 laid open on 

February 8, 2000), for allowing the reader to easily 
understand the present application, it will be 
described in detail. 

To generalize Expression (1) of the linear 
25 combination model defined by linear combination of 

predictive coefficients and pixel values for 
calculating a predictive value E , defining a matrix W 
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as a set of predictive coefficients Wj, a matrix X as a 
set of student data x^, and a matrix Y ? as a set of 
predictive values E [y t ] as Expression (3), an 
observation equation as Expression (4) is satisfied. 



X = 



(3) 











H, w- 




,r= 












H XW = 
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=3 In this case, a component x ± j of the matrix X 

-10 represents j-th student data of an i-th set of student 

data (a set of student data used for predicting i-th 

□ teacher data y ± ) ♦ A component w-j of the matrix W 

represents a predictive coefficient that is multiplied 
by j-th student data of a set of the student data. y ± 
15 represents i-th teacher data. Thus, E [yil represents 
a predictive value of i-th teacher data. In other 
words, y on the left side of Expression (1) is obtained 
by omitting the suffix i of the component y ± of the 
matrix Y. x z , x 2 , of the right side of Expression 

20 (1) are obtained by omitting the suffix i of the 

component x i:J of the matrix X. 
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Now, by applying the method of least squares 
to the observation equation, a predictive value E [y] 
close to a pixel value y of an HD picture is obtained. 
In this case, when the matrix E as a set of residuals e 
of a predictive value E [y] to a pixel value y of an HD 
pixel is defined with Expression (5), a residual 
formula as Expression (6) is satisfied with Expression 
(4). 



rJLO 
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E = 

















(5) 



XW = Y + E ... (6) 

In this case, the predictive coefficient w± 
for obtaining the predictive value E [y] close to a 
pixel value y of an HD picture is obtained by 
minimizing the square error of Expression (7). 



(7) 



Thus, when the result of which the square 
error is differentiated with the predictive coefficient 

becomes 0, (in other words) the predictive 
coefficient Wj that satisfies Expression (8) becomes an 
optimum value for obtaining the predictive value E [y] 
that is close to the pixel value y of an HD pixel. 



de 1 
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2 + _ + ^_^_ = o(y =X2,.„,7) ... (8) 
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Thus, when Expression (6) is differentiated 

with the predictive coefficient w 3 , Expression (9) is 
satisfied. 

— = JT 42f ..^S_ 1,2,..., J) ... (9) 

With Expression (8) and Expression (9), 
Expression (10) is obtained. 

J*/** = °' J***" - <W, " 0 - . . ( 10 ) 

Considering the relation among the student 
data Xij, predictive coefficients Wj, teacher data y i# 
and residuals e-j of the residual equation as Expression 
(6), with Expression (10), the normal equations of 
Expression (2) can be obtained. In this case, for each 
normal equation as Expression (2), it is preferred to 
prepare a predetermined number of sets of student data 
Xij and teacher data y ± . Thus, the normal equations 
corresponding to the number J of predictive 
coefficients Wj can be obtained. By solving Expression 
(2), an optimum predictive coefficient w 3 can be 
obtained. 

Fig. 12 shows an example of the structure of 
the data processing apparatus 1 that performs another 
processing method for increasing the resolution in the 
process mode (1). In the other processing method for 
the process mode (1), the post-processing portion 14 
selects a scanning direction converting portion 14B. 



The function controlling portion 11 outputs a 
control signal corresponding to a command that is input 
in this case to the data processing portion 12. 

The data processing portion 12 is initialized 
5 with a control signal supplied from the function 

controlling portion 11. Thereafter, the number of 
pixels in the horizontal direction of an SD picture as 
input data supplied from an external device is doubled. 
Thereafter, a horizontal scanning sequence (television 
10 raster scanning sequence) is converted into a vertical 

3 scanning sequence. Moreover, the number of pixels in 

3 the vertical direction is doubled. In other words, in 

3 the case, an HD picture with a higher resolution than 

j an SD picture is generated. 

15 However, since picture data that is output 

from the data processing portion 12 is still in the 
i vertical scanning sequence, it is necessary to restore 

^ "the picture data to the horizontal scanning sequence. 

Thus, in the data processing apparatus 1, the post- 
20 processing portion 14 disposed downstream of the data 
processing portion 12 selects the scanning direction 
converting portion 14 IB corresponding to a control 
signal supplied from the function controlling portion 
11. 

25 As with the intermediate data processing 

portion 23 of the data processing portion 12, the 
scanning direction converting portion 141B has a memory. 
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By controlling writing and/or reading process of the 
picture data that is supplied from the data processing 
portion 12 to/from the memory, the vertical scanning 
sequence is converted into the horizontal scanning 
5 sequence. For example, picture data that is supplied 

from the data processing portion 12 to the scanning 
direction converting portion 141B is address -controlled 
so that pixels are written in the vertical scanning 
. direction to the memory. When picture data is read 
0 from the memory, the memory is address -controlled so 

that pixels are read in the horizontal scanning 
direction. Thus, the scanning direction of data is 
converted from the vertical scanning direction to the 
horizontal scanning direction. The picture data 
5 restored to the horizontal scanning sequence by the 

scanning direction converting portion 141B is output as 
output data to an external device. 

Fig. 13 shows an example of the connections 
of the data processing portion 12 that performs the 
0 other processing method in the process mode (1). In 

this example, in the input data processing portion 21, 
one classifying adaptive processing portion 31-1 is 
used. SD picture data that is input as input data to 
the data processing apparatus 1 is supplied to the 
> classifying adaptive processing portion 31-1. 

The classifying adaptive processing portion 
31-1 performs a classifying adaptive process for 
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generating a picture whose number of pixels in the 
horizontal direction is twice as many as the supplied 
SD picture data (hereinafter, sometimes, the generated 
picture is referred to as intermediate picture) and 
supplies picture data of the intermediate picture 
(hereinafter, sometimes, the picture data is referred 
to as intermediate picture data) to the selector 22. 

In the example, in the intermediate data 
processing portion 23, the memory 41-1 is used. The 
intermediate picture data is supplied from the select 
portion 22 to the memory 41-1. The intermediate data 
processing portion 23 address -controls the memory 41-1 
so that the data written in the horizontal scanning 
direction to the memory 41-1 is read in the vertical 
scanning direction therefrom. Thus, the intermediate 
data processing portion 23 converts the horizontal 
scanning sequence of the intermediate picture into the 
vertical scanning sequence and returns the converted 
picture to the selector 22 . 

In this example, in the output data 
processing portion 24, the classifying adaptive 
processing portion 51-1 is used. The intermediate 
picture data is supplied from the selector 22 to the 
classifying adaptive processing portion 51-1. The 
classifying adaptive processing portion 51-1 performs a 
classifying adaptive process for doubling the number of 
pixels in the vertical direction of the intermediate 
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picture corresponding to the intermediate picture data 
supplied from the select portion 22 and supplies the 
generated picture data to the post -processing portion 
14 disposed downstream of the classifying adaptive 
processing portion 51-1. 

The picture data supplied to the post- 
processing portion 14 is supplied to the scanning 
direction converting portion 14 IB selected by the 
selector 140 corresponding to a control signal supplied 
from the function controlling portion 11. The scanning 
direction converting portion 141B performs a write 
control and a read control for the memory so that 
picture data that is written in the vertical scanning 
direction to the memory is read in the horizontal 
scanning direction therefrom- As a result, the 
vertical scanning sequence of the picture data is 
restored to the horizontal scanning sequence. The 
picture (HD picture) restored to the horizontal 
scanning sequence by the scanning direction converting 
portion 14 IB is output as output data to an external 
device . 

Next, the predictive process of the 
classifying adaptive processing portion 31-1 for the 
input data processing portion 21 according to the other 
processing method in the process mode (1) will be 
described. 

The class tap extracting circuit 71 (see Fig. 
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8) of the classifying adaptive processing portion 31-1 
reads class tap extraction information required in this 
example from the register 63A of the register group 63, 
forms class taps corresponding to the class tap 
5 extraction information, and supplies the class taps to 

the classifying circuit 72. 

Fig. 14 shows an example of class tap 
extraction information corresponding to the other 
processing method in the process mode (1), Fig. 14 
10 shows an example of class tap extraction information in 

:) the process for doubling the number of pixels in the 

3 horizontal scanning direction in the process mode (1). 

3 In Fig. 14, • (black circle) and O (white circle) 

represent SD pixels. Assuming that a considered pixel 
15 is a black circle pixel, class taps are composed of 

~ seven pixels arranged on a line of which the considered 

i pixel is placed at the center thereof. On the other 

J hand, predictive taps that will be described later are 

composed of 11 pixels that are wider than the class 
20 taps by two pixels. 

The class taps and predictive taps used for 
the process for doubling the number of pixels in the 
vertical direction may be arranged in the vertical 
direction rather than the horizontal direction shown in 
25 Fig* 14. 

Next, with reference to Fig. 15, this process 
will be described in detail. It is assumed that lines 



40 



K-l to K+l are arranged in a predetermined field of an 
SD picture as input data. In Fig. 15, large circles O 
represent SD pixels. Small circles represent pixels of 
an HD picture whose number of pixels in the horizontal 
5 direction is twice as many as an SD picture. The 

pixels axe arranged as input data on lines K-l to K+l 
of a predetermined field of the SD picture as shown in 
Fig. 15 • 

Now, the case that pixel Y K , 5 of an 
10 intermediate picture on line K is a considered pixel is 

considered. In the example, according to the class tap 
i[ extraction information that is read from the register 

group 63A, a total of seven SD pixels that are SD pixel 
- s X K , 3 that is predicted as a pixel having the highest 

3L5 correlation to pixel Y K , 5 on line K, two SD pixels X K , 2 

* and X K ,i arranged on the left of SD pixel X K , 3 , two SD 

pixels X K ,4 and XK,5 arranged on the right of SD pixel 
-I X K , 3 , and SD pixels X k -i #3 and SD pixel X K+ i,3 arranged on 

line K-l and line K+l at the positions corresponding to 
20 SD pixel X K ,3 are selected as class taps. 

Before classifying the considered pixel, the 
classifying circuit 72 reads class tap extraction 
information required in the process mode from the 
register 63A of the register group 63, extracts class 
25 taps corresponding to the class tap extraction 

information, and classifies the considered pixel using 
the extracted class taps. The decided class of the 
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considered pixel is supplied as an address to the 
coefficient memory 83 of the adaptive processing 
portion 62. 

The predictive tap extracting circuit 81 of 
the adaptive processing portion 62 reads predictive tap 
extraction information required in the example from the 
register 63B of the register group 63, forms predictive 
taps corresponding to the predictive tap extraction 
information, and supplies the predictive taps to the 
predictive calculating circuit 82. 

The coefficient memory 83 reads coefficient 
information required in the example from the register 
63C of the register group 63. In the coefficient 
memory 83, a bank used corresponding to the predictive 
information is set. From the bank, a predictive 
coefficient set for doubling the number of pixels in 
the horizontal direction is read from predictive 
coefficient sets stored at an address corresponding to 
the class supplied from the classifying circuit 72. 
The predictive coefficient set that is read from the 
bank is supplied to the predictive calculating circuit 
82. 

The predictive calculating circuit 82 
calculates Expression (1) with the predictive 
coefficient set supplied from the coefficient memory 83 
and pixel values of SD pixels that compose predictive 
taps supplied from the predictive tap extracting 
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circuit 81 and treats the predictive value of the 
considered pixel as a pixel value of a pixel of the 
intermediate picture . The intermediate picture 
composed of the predicted pixels is supplied to the 
5 selector 22. 

Since the learning process according to the 
other processing method in the process mode (1) is 
basically the same as that of the data processing 
apparatus 1 in the process mode (1), the description 
10 thereof will be omitted. 

=5 Since the predictive process of the 

y classifying adaptive processing portion 51-1 of the 

:f output data processing portion 24 in the example of the 

other processing method in the process mode (1) is 
315 basically the same as that of the classifying adaptive 

« processing portion 31-1 of he input data processing 

y portion 21 of the process mode (1), the detailed 

5 description thereof will be omitted. The classifying 

portion 61 of the classifying adaptive processing 
20 portion 51-1 and the predictive tap extracting circuit 

81 and the coefficient memory 83 of the adaptive 
processing portion 62 extract class taps and predictive 
taps corresponding to tap extraction information, 
predictive tap information, and coefficient information 
25 required in the example of the other processing method 

in the process mode (1) and read a predictive 
coefficient set from the coefficient memory 83. 



Corresponding to the results , pixels for doubling the 
number of pixels in the vertical direction are 
predicted. 

In addition, since the learning process of 
the classifying adaptive processing portion 51-1 of the 
output data processing portion 24 according to the 
other processing method in the process mode (1) is 
basically the same as the learning process of the 
classifying adaptive processing portion 31-1 in the 
above-described process mode (1), the description 
thereof will be omitted. 

In the above description, the classifying 
adaptive processing portion 51-1 of the output data 
processing portion 24 executes the classifying adaptive 
process for generating a picture whose number of pixels 
in the vertical direction is doubled so as to generate 
an HD picture. However, the present invention is not 
limited to such an example. For example, as with the 
above-described process mode (1), according to the 
other processing method in the process mode (1), a 
classifying adaptive process for predicting pixels 
arranged on a line (line A) of which a line of an SD 
picture is present at the same positions as pixels that 
compose an HD picture and another classifying adaptive 
process for predicting pixels arranged on a line (line 
B) of which a line of an SD picture is not present at 
the same positions as pixels that compose an HD picture 
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can be independently executed- In this case, these 
classifying adaptive processes of two classifying 
adaptive processing portions (for example, the 
classifying adaptive processing portions 51-1 and 51-2) 
5 of the output data processing portion 24 can be 

executed. 

In the above-described classifying adaptive 
process is described in International Patent Laid-Open 
No. WO 96/07987 and the corresponding United State 
10 Patent No. 5,903,481 filed by the applicant of the 

^ present invention. 

Fig. 16 shows an example of the structure of 
:^ the data processing apparatus 1 that executes a process 

H for generating a picture dedicated for a right eye and 

C3L5 a picture dedicated for a left eye in the above- 

M described process mode (2)* In the process mode (2), 

flj in the post-processing portion 14, the multiplexing 

p portion 141C is used. 

r In the example, the data processing apparatus 

20 1 generates a picture dedicated for a left eye 

(hereinafter referred to as left-eye picture) and a 
picture dedicated for a right eye (hereinafter referred 
to as right -eye picture) with SD picture data as input 
data supplied from an external device and outputs 
25 picture data (hereinafter referred to as stereo picture 

data) of the multiplexed picture (hereinafter referred 
to as stereo picture) as output data to an external 
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device . 

In this case, it is assumed that the SD 
picture data as input data is a television broadcast 
signal that has been A/D converted with a sampling 
5 clock of 13.5 MHz. In other words # the size of the SD 

picture is around 720 pixels (horizontal) x 480 lines 
(vertical) per frame. 

In this example, a command that causes the 
process for generating a left-eye picture and a right - 
10 eye picture to be executed is input to the function 

£i controlling portion 11. The function controlling 

JS portion 11 outputs a control signal corresponding to 

the input command to the data processing portion 12 . 
f"~ The function controlling portion 11 also 

^-15 extracts a vertical synchronous signal and a horizontal 

^ synchronous signal from the television broadcast signal 

fU and controls the data processing portion 12 and the 

til 

q multiplexing portion 141C of the post -processing 

portion 14 corresponding to the timings thereof. 

20 After the data processing portion 12 is 

initialized with a control signal supplied from the 
function controlling portion 11, the data processing 
portion 12 generates a left-eye picture and a right-eye 
picture with an SD picture that is input as input data. 

25 The multiplexing portion 141C of the post- 

processing portion 14 disposed downstream of the data 
processing portion 12 multiplexes the left-eye picture 
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data and the right-eye picture data supplied from the 
data processing portion 12, generates stereo picture 
data, and outputs the generated stereo picture data as 
output data to an external device. 
5 Fig. 17 shows an example of the connections 

of the data processing portion 12 in the process mode 
(2). In the example, in the input data processing 
portion 21, the two classifying adaptive processing 
portions 31-1 and 31-2 are used. SD picture data that 

10 is input as input data to the data processing apparatus 

=1 1 is supplied to the classifying adaptive processing 

portions 31-1 and 31-2. 

The classifying adaptive processing portion 
31-1 performs a classifying adaptive process for 

=15 generating left -eye picture data with the supplied SD 

picture data. The classifying adaptive processing 

y portion 31-1 supplies the generated left-eye picture 

5 data to the selector 22. On the other hand, the 

classifying adaptive processing portion 31-2 performs a 

20 classifying adaptive process for generating right-eye 

picture data with the supplied SD picture data. The 
classifying adaptive processing portion 31-2 supplies 
the generated right-eye picture data to the selector 22. 

The selector 22 supplies the left -eye picture 

25 data and the right-eye picture data supplied from the 

classifying adaptive processing portions 31-1 and 31-2 
of the input data processing portion 21 to the post- 
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processing portion 14, In the post-processing portion 
14, the route of the selector 140 for the supplied 
left -eye picture data and right -eye picture data is 
controlled corresponding to a control signal supplied 
5 from the function controlling portion 11. The left-eye 

picture data and right -eye picture data are supplied to 
, the multiplexing portion 141C. 

In the process mode (2), the intermediate 
data processing portion 23 and the output data 
10 processing portion 24 are not used. 

O Next, the predictive process of the 

=.g classifying adaptive processing portion 31-1 of the 

P input data processing portion 21 in the example will be 

\,i described. 

The class tap extracting circuit 71 (see Fig. 
f" 8) of the classifying adaptive processing portion 31-1 

^ reads class tap extraction information required in the 

O example from the register 63A of the register group 63. 

The classifying adaptive processing portion 31-1 
20 designates each of pixels that compose the left -eye 

picture finally obtained in the classifying adaptive 
process as a considered pixel. The left -eye picture 
does not actually exist before the predictive process 
is performed. The classifying adaptive processing 
25 portion 31-1 extracts class taps corresponding to class 

tap extraction information corresponding to each 
considered pixel and supplies the extracted class taps 
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to the classifying circuit 72. 

Figs. 18A and 18B show examples of class tap 
extraction information in the process mode (2). Fig. 
18A shows class tap extraction information for a left- 
5 eye picture. Fig. 18B shows class tap extraction 

information for a right-eye picture. In Figs. 18A and 
18B, • (black circle) and O (white circle) represent 
SD pixels. Now, it is assumed that in a left-eye 
picture shown in Fig. 18 A, a considered pixel is a 
10 black circle. In this case, class taps are composed of 

ri 4 pixels x 3 pixels that contains a considered pixel 

j{ and that slightly deviate from the considered pixel (in 

the example, by 0.5 pixels). On the other hand, 
r! predictive taps (that will be described later) are 

HL5 composed of 7 pixels x 3 pixels that are wider to the 

H right than the class taps by one pixel and to the left 

fU than the class taps by two pixels. 

q Unlike the left-side picture, in the right- 

side picture shown in Fig. 18B, class taps are composed 

20 of 4 pixels x 3 pixels that slightly deviate from a 

considered pixel (by 0.5 pixels as with the left-eye 
picture). In the right-side picture, predictive taps 
are composed of 7 pixels x 3 pixels that are wider to 
the left of the class taps by one pixel and to the 

25 right of the class taps by two pixels. 

The classifying circuit 72 reads class tap 
extraction information required in the process mode (2) 
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from the register 63A of the register group 63, decides 
the class of the considered pixel corresponding to the 
class tap extraction information, and supplies the 
class to the coefficient memory 83 of the adaptive 
processing portion 62. 

The predictive tap extracting circuit 81 of 
the adaptive processing portion 62 reads predictive tap 
extraction information required in the process mode (2) 
from the register 63B of the register group 63. The 
predictive tap extracting circuit 81 extracts 
predictive taps from an SD picture supplied to the 
adaptive processing portion 62 for the considered pixel 
corresponding to the predictive tap extraction 
information that has been read and supplies the 
extracted predictive taps to the predictive calculating 
circuit 82. For example, as practically shown in Fig* 
19, predictive taps AR1 denoted by a dotted line are 
formed and supplied to the predictive calculating 
circuit 82. 

The coefficient memory 83 reads coefficient 
information required in the example from the register 
63C of the register group 63 and sets a bank that is 
used corresponding to the coefficient information. The 
coefficient memory 83 reads a predictive coefficient 
set for generating a left -eye picture from predictive 
coefficient sets stored at an address of the bank 
corresponding to the class supplied from the 
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classifying circuit 72 and supplies the predictive 
coefficient set to the predictive calculating circuit 
82. 

The predictive calculating circuit 82 
calculates Expression (1) with the predictive 
coefficient set supplied from the coefficient memory 83 
and pixel values of the SD picture that compose the 
predictive taps supplied from the predictive tap 
coefficient circuit 81 and designates the predictive 
value of the considered pixel as the pixel value of a 
pixel that composes the left -eye picture. 

The left -eye picture composed of pixels 
having pixel values that have been predicted in such a 
manner are supplied to the selector 22. 

Likewise, the classifying adaptive processing 
portion 31-2 processes a right-eye picture. Since the 
predictive process of the classifying adaptive 
processing portion 31-2 is basically the same as the 
predictive process of the classifying adaptive 
processing portion 31-1, the detailed description 
thereof will be omitted. The classifying portion 61 of 
the classifying adaptive processing portion 31-2, the 
predictive tap extracting circuit 81 of the adaptive 
processing portion 62, and the coefficient memory 83 
extract class taps corresponding to tap extraction 
information, predictive tap extraction Information, and 
coefficient information required in the process mode 



(2) and read a predictive coefficient set from the 
coefficient memory 83. 

For example, the predictive tap extracting 
circuit 81 forms predictive taps AR2 (denoted by a dot- 
dash line shown in Fig. 19) that deviate by several 
pixels to the right of the predictive taps AR1 formed 
by the classifying adaptive processing portion 31-1. 
The amount of deviation between the predictive taps AR1 
and AR2 can be set "corresponding to parallax designated 
between a left -eye picture and a right -eye picture that 
are generated. 

In addition, the coefficient memory 83 of the 
classifying adaptive processing portion 31-2 reads a 
predictive coefficient set for generating a right-eye 
picture from the bank and supplies the predictive 
coefficient set to the predictive calculating circuit 
82. The predictive calculating circuit 82 calculates 
Expression (1) with the predictive coefficient set 
supplied from the coefficient memory 83 and the 
predictive taps supplied from the predictive tap 
extracting circuit 81 and designates the predictive 
value of the considered pixel as the pixel value of a 
pixel that composes the right -eye picture. 

The right -eye picture composed of pixels 
having predicted pixel values is supplied to the 
selector 22. 

Next, the learning process in the process 
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mode (2) will be described. When the learning process 
is performed by the classifying adaptive processing 
portion 31-1, as shown in Fig. 20, an object is 
photographed by three television cameras (a left-eye 
5 camera L, a right -eye camera R, and a center camera C) 

disposed at positions having respective parallaxes. 
Among the three photographed pictures, pictures 
photographed by the left-eye camera L and the center 
camera C are used. However, in the learning process of 
10 the classifying adaptive processing portion 31-2, 

& pictures photographed by the right -eye camera R and the 

% % center camera C are used. 

^ Fig. 21 shows an example of the structure of 

H the classifying adaptive processing portions 31-1 and 

0.5 31-2 that perform a learning process in the process 

M mode (2). In the structure, the thin-out circuit 91 of 

ry the structure shown in Fig. 11 is not used. 

q A picture photographed by the center camera C 

is supplied to the classifying portion 92 and the 
20 predictive tap extracting circuit 93. A picture 

photographed by the left -eye camera L is supplied to 
the teacher data extracting circuit 95. In other words, 
in the example, the picture photographed by the center 
camera C is classified (namely, a class of the picture 
25 is decided) . With teacher data that is a picture 

photographed by the left -eye camera L, a predictive 
coefficient that represents the correlation between the 
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picture photographed by the center camera C and the 
picture photographed by the left-eye camera L is 
obtained for each class. 

A set of predictive coefficients that have 
5 been obtained in such a manner is stored to the 

coefficient memory 83 of the classifying adaptive 
processing portion 31-1. 

Since the processes of the classifying 
portion 92 to the calculating circuit 97 are basically 
10 the same as those in the case shown in Fig. 11, the 

y description thereof will be omitted. 

^ The learning process of the classifying 

O adaptive processing portion 31-2 is the same as that of 

|u the classifying adaptive processing portion 31-1 except 

3 15 that a picture photographed by the right-eye camera R 

is teacher data. In other words, a predictive 
}% coefficient that represents the correlation between the 

^ picture photographed by the center camera C and the 

picture photographed by the right -eye camera R is 
20 obtained. A set of predictive coefficients that have 

been obtained in such a manner is stored to the 
coefficient memory 83 of the classifying adaptive 
processing portion 31-2. 

The classifying adaptive process for 
25 generating a left-eye picture and a right-eye picture 

is described in Japanese Patent Laid-open Publication 
No. HEI 9-55962 (laid open on February 25, 1997) that 



was filed by the applicant of the present invention. 

Fig. 22 shows an example of the structure the 
data processing apparatus 1 that executes a process for 
generating a luminance signal and color difference 
5 signals in the process mode (3). In this example, an 

NTSC (National Television System Committee) format 
picture (hereinafter referred to as NTSC picture) is 
supplied from an external device to the data processing 
apparatus 1, In this example, it is assumed that the 
10 NTSC picture data supplied to the data processing 

apparatus 1 is only an SD picture as a so-called 
Z composite video signal of which a luminance signal Y 

^ and color difference signals U and V are frequency- 

^ multiplexed (hereinafter, the composite video signal is 

3.5 referred to as composite signal NTSC). 

* The data processing apparatus 1 separates the 

y supplied NTSC picture data into luminance Y and color 

? differences U and V. When the NTSC picture data is 

supplied to the data processing apparatus 1, the pre- 
20 processing portion 13 performs a gain compensating 

process, a subcarrier reproducing process, a phase 
shifting process, and so forth for the NTSC picture 
data and supplies the processed NTSC picture data to 
the data processing portion 12. The data processing 
25 portion 12 performs the classifying adaptive process 

for the supplied data corresponding to the pre-learnt 
predictive coefficient set, generates the luminance 
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signal Y and the color difference signals U and V, and 
outputs these signals* Hereinafter, these output 
signals are together referred to as component signal (Y, 
U, V). In the process mode (3), the pre-processing 
5 portion 13 is not the above -described delaying circuit, 

but a circuit that performs the gain compensating 
process, sub-carrier reproducing process, phase 
shifting process, and so forth. 

Fig. 23 shows an example of the connections 
10 of the data processing portion 12 in the process mode 

(3). In the process mode (3), in the input data 
y processing portion 21, the three classifying adaptive 

processing portions 31-1, 31-2, and 31-3 are used. 
U Signals NTSC-Y, NTSC-U, and NTSC-V of which the SD 

15 picture data that is input as input data to the data 

jl processing apparatus 1 is pre-processed in a 

q predetermined manner (that will be described later) are 

^ supplied to the classifying adaptive processing 

portions 31-1, 31-2, and 31-3, respectively. 
20 The classifying adaptive processing portion 

31-1 performs a classifying adaptive process for 
generating the luminance signal Y with the supplied 
signal NTSC-Y and supplies the generated luminance 
signal Y to the selector 22. Likewise, the classifying 
25 adaptive processing portions 31-2 and 31-3 perform 

classifying adaptive processes for generating the color 
difference signals U and V with the supplied signals 
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NTSC-U and NTSC-V, respectively, and supply the 
generated color difference signals U and V to the 
selector 22. 

The selector 22 supplies the luminance signal 
5 Y and the color difference signals U and V supplied 

from the classifying adaptive processing portions 31-1 , 
31-2, and 31-3 of the input data processing portion 21 
as output signals to the post-processing portion 14. 
The post -processing portion 14 outputs the supplied 
10 signals as they are. 

?M In the process mode (2), the intermediate 

% data processing portion 23 and the output data 

\H processing portion 24 are not used. 

Fig. 24 shows an example of the structure of 
QL5 the data processing apparatus 1 that accomplishes a Y/C 

H separating device that performs a classifying adaptive 

ry process for separating the composite signal NTSC into 

£=1 the component signal (Y, U, V) in the process mode (3). 

As denoted by a dot line shown in Fig. 24, a block 
20 disposed upstream of classifying circuits 50, 51, and 

52 corresponds to the pre-processing portion 13. A 
block disposed downstream of the classifying circuits 
50, 51, and 52 corresponds to the classifying adaptive 
processing portions 31-1, 31-2, and 31-3. A 
25 classifying portion 50 corresponds to the classifying 

portion 61 of the classifying adaptive processing 
portion 31. A predictive filter 53 corresponds to the 
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adaptive processing portion 62 of the classifying 
adaptive processing portion 31. In Fig. 24, the 
register group 63 of the classifying adaptive 
processing portion 31 is omitted. 
5 A composite signal NTSC is supplied from an 

input terminal 141. A sub-carrier reproducing circuit 
142 extracts a sub -carrier from the supplied composite 
signal NTSC and reproduces it. In addition, the 
supplied composite signal NTSC is supplied to gain 
10 compensating circuits 143, 144, and 145. The gain 

Q compensating circuit 143 performs a gain compensation 

a) for converting the level of the composite signal NTSC 

O into the level of the luminance signal Y. The signal 

j_j that is output from the gain compensating circuit 143 

"15 is the NTSC-Y signal and supplied to the classifying 

H circuit 150. 

V? Likewise, the gain compensating circuit 144 

p compensates the gain. A multiplying circuit 148 

multiplies a sub -carrier that has been phase -shifted by 
20 -123 degrees against the sub -carrier reproduced by the 

sub-carrier reproducing circuit 142 by the output of 
the gain compensating circuit 144 and thereby generates 
the NTSC-U signal. The NTSC-U signal is supplied to a 
classifying circuit 151. 
25 Likewise, the gain compensating circuit 145 

compensates the gain. A multiplying circuit 149 
multiplies a sub-carrier that has been phase-shifted by 
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-33 degrees against the sub-carrier reproduced by the 
sub -carrier reproducing circuit 142 by the output of 
the gain compensating circuit 145 and thereby generates 
the NTSC-V signal. The NTSC-V signal is supplied to a 
classifying circuit 152. 

The classifying circuit 150 outputs class 
code P and data B generated with the supplied NTSC-Y 
signal to a predictive filter 153. The predictive 
filter 153 reads a filter coefficient corresponding to 
the supplied class code P. The predictive filter 153 
calculates the filter coefficient and the supplied data 
B and thereby generates the luminance signal Y. The 
generated luminance signal Y is obtained from an output 
terminal 156. 

The classifying circuit 151 outputs class 
code P and data B generated with the supplied NTSC-U 
signal to a predictive filter 154. The predictive 
filter 154 reads a filter coefficient corresponding to 
the supplied class code P, calculates the filter 
coefficient and the supplied data B, and thereby 
generates the color difference signal U. The generated 
color difference signal U is obtained from an output 
terminal 157 . 

Likewise, the classifying circuit 152 outputs 
class code P and data B generated with the supplied 
NTSC-V signal to a predictive filter 155. The color 
difference signal is obtained from the predictive 
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filter 155 through an output terminal 158. In such a 
manner, the component signal (Y, U, V) can be obtained. 

Next, with reference to Figs* 25 and 26, 
class taps and predictive taps used in the process mode 
5 (3) will be described. As shown in Figs. 25A and 25B, 

when a considered pixel is pixel VO, class taps are 
composed of pixel VO, pixels VA and VB on the upper and 
lower lines of pixel VO at the corresponding positions 
of pixel VO in the field of pixel VO (0 field}, pixels 
10 VC and VD that are apart from pixel VO by one pixel in 

} the horizontal direction, and pixel VE of the preceding 

) frame (-1 frame = -2 frame) corresponding to pixel VO. 

3 The reason why pixels VC and VD that are apart from 

j pixel VO by one pixel in the horizontal direction are 

15 used is to synchronize the phase. 

I On the other hand, as shown in Fig. 26, 

i predictive taps are composed of pixel VO, pixels that 

? immediately surround pixel VO in the field of pixel VO, 

the above-mentioned pixels VC and VD, and all the 
20 corresponding pixels of the preceding frame of the 

field of pixel VO. Since a coefficient is obtained in 
consideration of the difference of phases of pixels in 
the learning process (that will be described later), 
predictive taps can be extracted from pixels that have 
25 different phases* 

Next, the learning process in the process 
mode (3) will be described. Fig. 27 shows an example 
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of the structure of the classifying adaptive processing 
portions 31-1, 31-2, and 31-3 that perform a learning 
process in the process mode (3). In the structure 
shown in Fig- 27, the thin-out circuit 91 of the 
structure shown in Fig. 11 is not used. Instead, an 
NTSC encoder 191 is disposed. In Fig. 27, similar 
portions to those in Fig. 11 are denoted by similar 
reference numerals and the detailed description thereof 
will be omitted. 

Predetermined luminance signal Y and color 
difference signals U and V are supplied to the NTSC 
encoder 191. The NTSC encoder 191 encodes the input 
signals into a composite signal NTSC. The composite 
signal NTSC is supplied to a classifying portion 92 and 
a predictive tap extracting circuit 93. In other words, 
in such a structure, with the composite signal NTSC, a 
considered pixel is classified and predictive taps are 
extracted. In addition, the luminance signal Y and 
color difference signals U and V are supplied to a 
teacher data extracting circuit 95. In the structure, 
with teacher signal that are the luminance signal and 
color difference signals U and V, using the composite 
signal NTSC, a predictive coefficient set is generated. 

A predictive coefficient set is generated for 
each of the luminance signal Y and color difference 
signals U and V. The predictive coefficient set 
corresponding to the luminance signal Y is applied for 



the classifying adaptive processing portion 31-1 
(namely, the classifying circuit 150 and the predictive 
filter 153). The predictive coefficient set 
corresponding to the color difference signal U is 
5 applied for the classifying adaptive processing portion 

31-2 {namely, the classifying circuit 151 and the 
predictive filter 154). The predictive coefficient set 
corresponding to the color difference signal V is 
applied for the classifying adaptive processing portion 

10 31-3 (namely, the classifying circuit 152 and the 

3 predictive filter 155). 

3 Fig. 28 shows an example of the structure of 

3 the data processing apparatus 1 that converts the 

j aspect ratio in the above -described process mode (4), 

"15 In this example, the data processing portion 12 

increases the number of pixels only in the horizontal 
% direction of a picture corresponding to picture data as 

3 input data (hereinafter referred to as input picture) 

and changes the aspect ratio. 
20 In this example, with three chronologically 

successive pixels (three pixels arranged successively 
in the horizontal direction), four chronologically 
successive pixels (four pixels arranged successively in 
the horizontal direction) of a picture (hereinafter 
25 referred to as output picture) corresponding to picture 

data as output data are predicted. In this case, 
pixels of an output picture are newly predictively 
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created. In other words, in this example, an output 
picture is generated in the state that the ratio of the 
number of pixels in the horizontal direction of the 
input picture to the number of pixel in the horizontal 
5 direction of the output picture is 3 to 4. 

The function controlling portion 11 outputs a 
control signal corresponding to an input command to the 
data processing portion 12. The data processing 
portion 12 selects a bank corresponding to the process 
10 mode (4) corresponding to the control signal supplied 

from the data processing portion 12. The data 
v 3 processing portion 12 executes a classifying adaptive 

u3 process for generating four pixels with three pixels 

corresponding to a coefficient set of the selected bank. 
C3L5 The data 'processing portion 12 performs the classifying 

M adaptive process for each of the four pixels. Thus, 

rn the selector 140 of the post-processing portion 14 

JSJ selects the multiplexing portion 141C that performs a 

f " multiplexing process corresponding to a control signal 

20 supplied from the function controlling portion 11. 

The multiplexing portion 141C multiplexes 
picture data supplied from the data processing portion 
12. In other words, the multiplexing portion 141C 
outputs an output picture of which the aspect ratio of 
25 the input picture was changed as output data to an 

external device. 

Fig. 29 shows an example of the connections 
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of the data processing portion 12 in the process mode 
(4). In this example, in the input data processing 
portion 21, the four classifying adaptive processing 
portions 31-1 to 31-4 are used. An input picture is 
5 supplied to the classifying adaptive processing 

portions 31-1 to 31-4. 

The input data processing portion 21 performs 
a predetermined classifying adaptive process for the 
input picture and outputs picture data of which the 

10 ratio of the number of pixels in the horizontal 

i direction of an input picture to the number of pixels 

in the horizontal direction of an output picture is 3 

} to 4 to the selector 22. 

The selector 22 supplies picture data 

15 supplied form the input data processing portion 21 to 

the post-processing portion 14* As was described above, 
in the post-processing portion 14, the selector 140 

f selects the multiplexing portion 141C corresponding to 

a control signal supplied form the function controlling 

20 portion 11. The picture data supplied to the post- 

processing portion 14 is supplied to the multiplexing 
portion 141C. The multiplexing portion 141C 
multiplexes the picture data supplied from the post- 
processing portion 14 and outputs the multiplexed 

25 picture data as output picture data. 

The intermediate data processing portion 23 
and the output data processing portion 24 are not used 
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in the process mode (4). 

Next, with reference to Fig. 30, the 
predictive process of the classifying adaptive 
processing portion 31 of the input data processing 
5 portion 21 in the process mode (4) will be described. 

Figs. 30A and 30B schematically show pixels of an input 
picture and pixels of an output picture. 

The classifying adaptive processing portion 
31-1 performs a classifying adaptive process in such a 
10 manner that a pixel of an output picture that has a 

^ first phase relation to a pixel of an input picture 

= (namely, a pixel arranged on the same vertical line as 

a pixel of an input picture) is designated as a 
* considered pixel and predicts the pixel value of the 

3L5 considered pixel. In the example shown in Fig. 30, 

^ pixels Pbl and Pb5 that compose an output picture and 

!j pixels Pal and Pa4 that compose an input picture and 

~ that correspond to pixels pbl and pb5 are used as 

considered pixels. 
20 The classifying adaptive processing portion 

31-2 performs a classifying adaptive process in such a 
manner that a pixel that is immediately chronologically 
preceded by a pixel whose pixel value has been 
predicted by the classifying adaptive processing 
25 portion 31-1 (namely, a pixel having a second phase 

relation to a pixel of an input picture) is designated 
as a considered pixel and predicts the pixel value of 
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the considered pixel. In the example shown in Fig. 30B, 
pixel Pbl is a pixel whose pixel value is predicted by 
the classifying adaptive processing portion 31-1. 
Pixel Pb2 arranged adjacent to pixel Pbl is a 
5 considered pixel designated by the classifying adaptive 

processing portion 31-2. 

The classifying adaptive processing portion 
31-3 perforins a classifying adaptive process in such a 
manner that a pixel that is immediately chronologically 
10 preceded by a pixel whose pixel value is predicted by 

the classifying adaptive processing portion 31-2 
(namely, a pixel having a third phase relation to a 
O pixel of an input picture) is designated as a 

considered pixel and predicts the pixel value of the 
^15 considered pixel. In the example shown in Fig. 30B, 

L pixel Pb2 is a pixel whose pixel value is predicted by 

yi the classifying adaptive processing portion 31-2. 

J J Pixel Pb3 arranged adjacent to pixel Pb2 is a 

considered pixel designated by the classifying adaptive 
20 processing portion 31-3. 

The classifying adaptive processing portion 
31-4 performs a classifying adaptive process in such a 
manner that a pixel (having a fourth phase relation to 
a pixel of an input picture) that is immediately 
25 chronologically preceded by a pixel (in the example 

shown in Fig* 30B, pixel Pb3) whose pixel value is 
predicted by the classifying adaptive processing 
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portion 31-3 is designated as a considered pixel and 
predicts the pixel value of the considered pixel. In 
the example shown in Fig, 30B, pixel Pb3 is a pixel 
whose pixel value is predicted by the classifying 
adaptive processing portion 31-3. Pixel Pb4 arranged 
adjacent to pixel Pb3 is a considered pixel designated 
by the classifying adaptive processing portion 31-4. 

In other words, since the number of pixels in 
the horizontal direction is increased in the ratio of 3 
to 4, there are four types of phase relations in the 
horizontal relation of a pixel (considered pixel) of an 
output picture and a pixel of an input picture. Each 
of the classifying adaptive processing portions 31-1 to 
31-4 performs a classifying adaptive process 
corresponding to one of the four types of phase 
relations and supplies the generated pixel to the 
selector 22. 

The selector 22 supplies picture data (pixel) 
supplied from the classifying adaptive processing 
portion 31 to the post -processing portion 14. As was 
described above, in the post-processing portion 14, the 
selector 14 supplies the supplied picture data to the 
multiplexing portion 14 1C corresponding to a control 
signal supplied from the function controlling portion 
11. The multiplexing portion 141C multiplexes the 
picture data and outputs the multiplexed picture data 
to an external device. In such a manner, the aspect 
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ratio is changed. 

In the process mode (4), by increasing the 
number of pixels in the horizontal scanning direction, 
the aspect ration is changed. Likewise, by decreasing 
5 the number of pixels in the horizontal scanning 

direction, the aspect ratio can be changed. 

Fig. 31 shows an example of the structure of 
the data processing apparatus 1 that generates a 
picture having a different resolution in the above- 

10 described process mode (5). In the example, the data 

processing portion 12 generates picture data 
corresponding to a plurality of pictures having 
different resolutions with SD picture data as input 
data supplied from an external device and outputs the 

15 generated picture data as output data to an external 

device . 

Fig. 32 shows an example of the connections 
of the data processing portion 12 in the process mode 
(5). In the example, in the input data processing 

20 portion 21, the classifying adaptive processing 

portions 31-1 and 31-2 are used. SD picture data as 
input data is supplied to the classifying adaptive 
processing portions 31-1 and 31-2. 

The classifying adaptive processing portion 

25 31 performs a classifying adaptive process for supplied 

SD picture data and supplies the generated picture data 
having a first resolution to the selector 22. 
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The classifying adaptive processing portion 
31-2 performs a classifying adaptive process for the 
supplied SD picture data and supplies the generated 
picture data having a second resolution to the selector 
5 22. 

The selector 22 supplies picture data having 
the first resolution supplied from the classifying 
adaptive processing portion 31-1 of the input data 
processing portion 21 to the classifying adaptive 
10 processing portions 51-1 and 51-2 of the output data 

processing portion 24. 

The selector 22 supplies the picture data 
55 having the second resolution supplied from the 

M classifying adaptive processing portion 31-2 of the 

□L5 input data processing portion 21 to the classifying 

u adaptive processing portions 51-3 and 51-4 of the 

hi output data processing portion 24. 

j-J In this example, in the output data 

processing portion 24, the classifying adaptive 
20 processing portions 51-1 to 51-4 are used. 

The classifying adaptive processing portion 
51-1 performs a classifying adaptive process for the 
picture data having the first resolution supplied 
through the selector 22 and generated by the 
25 classifying adaptive processing portion 31-1 and 

outputs the generated picture data having the third 
resolution as output data. 
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The classifying adaptive processing portion 
51-2 performs a classifying adaptive process for the 
picture data having the first resolution supplied 
through the selector 22 and generated by the 
5 classifying adaptive processing portion 31-1 and 

outputs the generated picture data having the fourth 
resolution as output data. 

The classifying adaptive processing portion 
51-3 performs a classifying adaptive process for the 
10 picture data having the second resolution supplied 

;3 through the selector 22 and generated by the 

J) classifying adaptive processing portion 31-2 and 

;3 outputs the generated picture data having the fifth 

j generation as output data. 

Is The classifying adaptive processing portion 

7 51-4 performs a classifying adaptive process for the 

Jf picture data having the second resolution supplied 

through the selector 22 and generated by the 
classifying adaptive processing portion 31-2 and 
20 outputs the generated picture data having the sixth 

resolution as output data. 

In the process mode (5), a plurality of 
pictures having different resolutions can be generated* 
Thus, for example, multi-windows having different sizes 
25 can be displayed. 

The classifying adaptive process of each 
classifying adaptive processing portion of the input 
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data processing portion 21 and the output data 
processing portion 24 may be the same as that of the 
above-described data processing apparatus 1 in the 
process mode (1). In addition, in the process mode (5), 
5 the same class taps and predictive taps as those in the 

above -described process mode (1) can be used. 

Fig. 33 shows an example of the structure of 
the data processing apparatus 1 that converts the frame 
rate in the above -de scribed process mode (6). In this 
10 case, the data processing apparatus 1 converts the 

frame rate of a picture that is input as input data. 
^ The function controlling portion 11 supplies 

5jJ a control signal corresponding to a command that is 

H input in the process mode (6) to the data processing 

p.5 portion 12. The data processing portion 12 selects a 

ik-i bank for the process mode (6) corresponding to the 

hi control signal supplied from the function controlling 

™ portion 11. The data processing portion 12 executes a 

r " classifying adaptive process for generating a new frame 

20 with picture data as input data corresponding to a 

coefficient set of the selected bank- In the process 
mode (6), the post-processing portion 14 selects the 
multiplexing portion 141C corresponding to the control 
signal supplied from the function controlling portion 
25 11 so as to multiplex a frame that is newly generated 

by the data processing portion 12. 

A multiplexing portion 106 chronologically 
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multiplexes picture data (as frames) supplied from the 
data processing portion 12 and outputs the multiplexed 
data as output data to the outside. 

Fig. 34 shows an example of the connections 
5 of the data processing portion 12 in the process mode 

(6). In the example, in the input data processing 
portion 21, the classifying adaptive processing 
portions 31-1 and 31-2 are used. A picture as input 
data is input to each of the classifying adaptive 
10 processing portions 31-1 and 31-2. 

As shown in Fig. 35A, it is assumed that the 
Q input picture has a frame structure with a period of Ta. 

3 On the other hand, as shown in Fig. 35B, it is assumed 

y that an output picture whose frame rate has been 

15 converted has a frame structure with a period of Ta/2. 

In other words, in this example, the frame rate is 
% doubled . 

- In the classifying adaptive processing 

portion 31-1, the multiplex portion 141C performs a 

20 classifying adaptive process for generating a frame of 

an output picture that is chronologically preceded by a 
frame that composes an input picture. In reality, the 
classifying adaptive processing portions 31-1 and 31-2 
treats a frame in the unit of a pixel. In the 

25 classifying adaptive process of the classifying 

adaptive processing portion 31-1, a frame that composes 
an output picture is generated. In the example shown 



in Fig. 35, as shown in Fig. 35B, frames B-2, B-4, and 
B-6 (white frames in Fig. 35B) of an output picture 
chronologically followed by frames A-l to A- 3 of an 
input picture shown in Fig. 35A are generated by the 
5 classifying adaptive processing portion 31-1 (see Fig. 

35B) . The classifying adaptive processing portion 31-1 
supplies the generated frames to the selector 22. 

The classifying adaptive processing portion 
31-2 performs a classifying adaptive process for 
10 generating a frame that composes an output picture and 

that is chronologically followed by a frame that 
composes an input picture by the multiplexing portion 

n 

n 141C disposed downstream thereof. In the classifying 

adaptive process of the classifying adaptive processing 
;JL5 portion 31-2, another frame that composes the output 

^ picture is generated. In the example shown in Fig. 35, 

f= as shown in Fig. 35B, frames B-l, B-3, and B-5 (hatched 

!: frames shown in Fig. 35B) of an output picture 

~ chronologically followed by frames A-l to A- 3 of an 

20 input picture shown in Fig. 35A are generated (see Fig. 

35B) . The classifying adaptive processing portion 31-2 
supplies the generated frames to the selector 22. 

The selector 22 supplies the frames supplied 
from the classifying adaptive processing portions 31-1 
25 and 31-2 to the post-processing portion 14. The post- 

processing portion 14 supplies the frames selected by 
the selector 140 corresponding to the control signal 
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supplied from the function controlling portion 11 to 
the multiplexing portion 141C. The multiplexing 
portion 141C multiplexes the supplied frames in the 
rule as was described with reference to Fig. 35B and 
5 outputs the multiplexed data as output data to an 

external device. In the process mode (6), in such a 
manner, the frame rate is converted. 

Fig. 36 shows an example of class tap 
extraction information in the process mode (6). Next, 

Q 

10 the class tap extraction information will be described 

O with reference to Fig. 35A. In frame A- 2 that contains 

*2 a considered pixel, 3 pixels x 3 pixels whose center 

O pixel is the considered pixel are extracted as class 

\,j taps. In frames A-l and A-3 that are immediately 

^15 preceded and followed by frame A- 2, pixels 

H corresponding to those extracted as the class taps in 

frame A- 2 are selected as class taps. In other words, 
O in the process mode (6), class taps are composed of 3 

pixels x 3 pixels x 3 frames. 
20 Predictive taps (not shown) include for 

example frame A- 2 that has a considered pixel. A total 
of five frames composed of frame A-2, two frames 
preceded by frame A-2, and two frames followed by frame 
A-2 are . extracted . In each of the five frames, 5 
25 pixels x 5 pixels wider than class taps by one pixel 

are extracted. In other words, in the process mode (6), 
predictive taps are composed of for example 5 pixels x 



5 pixels x 5 frames . 

Tlie classifying adaptive process of the 
classifying adaptive processing portions 31-1 and 31-2 
of the input data processing portion 21 may be the same 
as that of the data processing apparatus 1 in the 
process mode (1). 

In the above description, an example of which 
the present invention is applied to picture data was 
explained- However, the present invention is not 
limited to such an example. In other words, the 
present invention can be applied to other data such as 
audio data . 

Moreover, in the above description, the data 
processing apparatus 1 accomplishes the above-described 
process modes (1) to (6). However, the present 
invention is not limited to such an example. In other 
words, in the data processing apparatus 1, with a 
predetermined predictive coefficient set, a 
predetermined class tap structure, and a predetermined 
predictive tap structure, processes other than the 
process modes (1) to (6) can be accomplished. 



